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ABSTRACT 


The  Maximum  Likelihood  Method  and  the  Maximum  Entropy  Method  of  spectral 
estimation  are  described  and  interpreted  in  terms  of  the  innovation  filter  concept. 
Algorithms  are  developed  for  mapping  an  observed  data  sequence  into  the  spectral 
estimates.  The  resolution  performance  of  these  spectral  estimation  algorithms  is 
compared  by  applying  each  to  simulated  data  containing  two  tones  in  white  noise . 
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I. 


INTRODUCTION 


A.  Problem  Statement 

We  suppose  that  one  observes  a segment,  r(t)  0 st  sT,  of  a sample 
function  from  a zero-mean  stationary  random  process  and  wishes  to  generate  an 
estimate  of  the  power  density  spectrum.  When  it  is  desired  to  distinguish  between 
sharply  peaked  components  of  the  spectrum  at  some  minimal  separation,  A^ , then  the 
choice  of  a spectral  estimate  is  largely  dependent  on  the  time-bandwidth  product, 

W = TAf.  If  W>  > 1 then  any  one  of  a large  number  of  schemes  will  achieve  the  de- 
sired resolution  with  a reasonably  small  estimate  variance.  In  many  situations,  how- 
ever, the  observation  interval  is  constrained  to  be  relatively  short  (e.  g.  when  r(t)  may 
only  be  considered  stationary  over  a short  time  interval)  and  one  must  select  an 
estimate  subject  to  the  requirement,  W ~ 1. 


B.  The  Conventional  Method 

(11) 

When  the  conventional  method  (Blackman- Tukey  1959)  is  employed, 

m 

the  observed  data  is  used  to  obtain  the  covariance  estimate 


K (t)  = (T  - t)"1  \ r(t)  r*  (t  - t)  dt  = K (-t) 

JT 

0 <:  T s T £ T 
m 


(i.i) 


Where  Tm  is  the  maximum  lag  used.  The  spectral  estimate  is  the  Fourier  transform 
of  the  product,  K(t)  d (t),  where  d (t)  is  a given  symetric  "lag  window"  which 
vanishes  for  | t I > Tm 


1 


* 

Since  K (t)  is  an  unbiased  estimate  of  the  covariance,  E [ r (t)  r (t-T)  ] , 
the  average  estimated  spectrum  is  the  convolution  of  the  true  spectrum  with  the 
spectral  window. 


D (f) 


A 


dT  . 


Therefore,  our  high  resolution  (small  bias)  requirement  translates  into  a narrow 
band  constraint  on  the  spectral  window. 

Unfortunately,  the  estimate  variance  is  approximately  inversely 
proportional  to  the  spectral  window  bandwidth.  In  addition,  narrow  band  windows 
generally  have  significant  sidelobes  which  may  produce  spurious  peaks  in  the  estimate. 
We  are  thus  left  with  the  likelihood  that  there  is  no  spectral  window  which  provides 
satisfactory  performance  for  the  situation  of  interest. 

C.  The  Data  Adaptive  Methods 

Recently,  two  spectral  estimation  methods  have  appeared  in  the 

literature  which  give  us  a promising  alternative  approach  to  this  problem.  These 

* 

are  the  "data  adaptive"  methods  - the  M.  L.  M.  (Maximum  Likelihood  Method)  and 
the  M.  E.  M.  (Maximum  Entropy  Method).^  Similar  to  the  conventional  method,  each 
of  the  data  adaptive  methods  consists  of  a procedure  for  obtaining  the  spectral 
estimate  from  an  estimate  of  the  covariance,  K (T),  't1  £ . The  new  methods, 

however,  relinquish  the  linearity  and  simplicity  of  description  associated  with  the 


*The  M.  L.  M.  was  developed  for  application  to  frequency -wavenumber  spectral 
analysis  by  Capon,  [ Reference  6] , and  applied  to  time  series  analysis  by  Lacoss, 

| Reference  2] . 

t The  M.  E.  M.  and  its  application  to  time  series  analysis  first  appeared  in  a paper 
by  Burg,  [ Reference  7] . 

fWe  distinguish  here  between  K (t),  which  is  any  estimate  of  the  covariance,  and  the 
particular  estimate,  K (t). 
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conventional  method  in  an  attempt  to  provide  improved  estimate  performance  when 
high  resolution  is  required. 

For  convenience,  we  give  a short  description  of  the  methods  here.  A 
more  complete  description  is  contained  in  Chapter  II. 

The  M.  L.  M.  spectral  estimate  at  the  frequency  f Hz  is  obtained  as 
the  average  output  power  of  a particular  causal  convolutional  filter  of  duration  Tm 
seconds  when  the  input  is  a random  process  with  covariance  K (t)  for  lTl  5 Tm  . 
The  filter  is  designed  to  pass  the  power  in  a narrow  band  about  f Hz  and  minimize 
the  power  due  to  other  (interfering)  spectral  components.  The  average  output  power 
of  the  filter  is  given  by 


where  we  have  assumed  the  existance  of  the  inverse  kernel,  Q (t,  u),  of  K (t) 
defined  by  the  integral  equation 


(1.2) 


'0 


Q(t,  u)  K(u  - x)  du  = 6 (t  - x)  ; 0 < t,  x < Tm 


To  describe  the  M.  E.  M. , we  assume  that  the  measured  covariance  is 


of  the  form 


and  consider  the  causal  convolutional  filter  of  duration  T seconds  which  is  obtained 

m 

as  the  solution  to  the  integral  equation 


T 

N h(x:  T ) + h(u:  T ) p(x  - u)  du  = 0(x)  ; 0 £ x s T (1.3) 

o m j_  m m 

0 

The  M.  E.  M.  spectral  estimate  is  given  by  the  Fourier  transform  of  a certain  positive 
definite  extension  of  the  measured  covariance.  The  extension  is 

Ke(t)  =No6(x)+0e(x)  = KeM-t)  ; M <oc 


Where  3^  (x)  is  the  solution  to  the  Wiener-Hopf  integral  equation: 


T 

Nn  h(T:  Tm>  + ' m h(U:  Tm>  MT  ’ U>  du  = MT> 

o m m n c. 


0 s X <oo  (1.4) 


It  can  be  shown  from  1.4  that  the  filter,  6(T)  - h(x;  T ),  whitens  a process  with 
covariance  K„  (T)  and  the  spectral  height  of  the  white  output  is  N . Therefore,  the 
M.  E.  M.  spectral  estimate  is  given  by 


P (f:  T 
E m 


A 
) = 


T 

fm  , , ^ . -j2nfx 

\ h(x:  T ) e 
\ m 


dx 


-2 

(1.5) 
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The  M.  E.  M.  extension  of  the  measured  covariance  is  developed  in 
Chapter  II  by  using  the  concept  of  an  "innovation  process".  Other  interpretations* 
of  the  extension  appear  in  References  7 and  9. 

We  note  that  each  of  the  data  adaptive  methods  attempts  to  improve  on 
the  conventional  method  in  a different  manner.  The  M.  L.  M.  employs  a "sliding" 
bandpass  filter  which  adjusts  itself  to  the  random  process  under  consideration  in 
such  a way  that  the  spectral  estimate  at  one  frequency  is,  in  a sense,  least  affected 
by  spectral  components  at  other  frequencies.  The  M.  E.  M.  extrapolates  the  measured 
covariance  beyond  the  largest  available  lag  in  a unique  way  which  depends  on  the 
measured  covariance. 

D.  Performance  Considerations 

At  the  present  time,  a complete  evaluation  of  performance  for  the  data 
adaptive  methods  has  not  been  accomplished.  In  order  to  render  this  evaluation, 
one  must  necessarily  specify  a scheme  for  obtaining  the  measured  covariance  from 
the  observed  data  and  compute  the  bias  and  variance  of  the  spectral  estimate  for 
arbitrary  spectra  (usually  by  assuming  that  the  observed  process  is  Gaussian). 

The  complex  nature  of  the  new  methods  has  thus  far  hindered  this  computation  and 
the  current  results  are  restricted  to  the  uninteresting  case  where  W >>1.  t 

A partial  characterization  of  performance  may  be  obtained  by  applying 
the  estimation  methods  to  given  covariance  functions.  That  is,  we  take  the  measured 
covariance  to  be  a finite  segment  of  a known  covariance  function  and  evaluate  the 
estimate  performance  in  terms  of  the  largest  lag  used  and  the  covariance  parameters. 
The  obvious  limitation  of  this  approach  is  that  it  yields  no  information  about  estimate 
variance.  It  does,  however,  provide  information  which  is  roughly  equivalent  to 

*TTiese  employ  the  discrete  form  of  the  M.  E.  M.  which  is  indicated  in  Chapter  II. 

| These  results  are  discussed  by  Lacoss  in  Reference  2. 


5 


that  afforded  by  a bias  computation  and  is  therefore  useful  as  a basis  for  comparing 
the  resolution  obtainable  with  the  different  estimation  methods.* 

In  Chapter  III  we  employ  this  approach  to  obtain  a measure  of  resolution 
for  the  data  adaptive  methods.  This  is  done  by  taking  the  known  covariance  to  be 
from  a random  process  which  consists  of  two  equal  power  tones  in  white  noise  and 
computing  the  minimal  signal- to- noise  ratio  necessary  to  resolve  the  tones  as  a 
function  of  the  time -bandwidth  product.  The  computation  provides  a useful  standard 
of  comparison  for  the  data  adaptive  and  conventional  methods.  We  note,  however, 
that  this  measure  of  resolution  is  somewhat  simplified  in  that  it  does  not  directly 
relate  to  the  situations  where  the  tones  are  of  unequal  power  or  other  (interfering) 
spectral  components  are  present.  We  have  therefore  included  for  reference  a com- 
putation of  the  spectral  estimates  for  the  general  case  where  n tones  of  unequal 
power  are  superimposed  on  white  noise. 

E.  Implementation  of  the  Data  Adaptive  Methods 

Tiie  digital  computation  of  the  data  adaptive  spectra  from  the  observed 
segment,  r (t),0  ^t  ^T,  is  considered  in  Chapter  IV  where  we  use  discrete  forms 
of  the  two  methods  developed  in  Chapter  II.  We  indicate  alternative  schemes  for 
estimating  the  covariance  from  the  observed  data  and  develop  algorithms  for 
mapping  the  observed  data  into  the  spectral  estimate. 

In  Chapter  V we  employ  three  such  schemes  with  each  of  the  data 
adaptive  methods  and  apply  the  six  spectral  estimates  to  data  acquired  by  numerically 
simulating  two  tones  in  white  noise.  The  simulations  enable  us  to  provide  a quali- 
tative comparison  of  the  estimate  performances. 


*Some  comparisons  of  this  type  have  been  made  by  Lacoss  [ Reference  2]  . 
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II.  THE  DATA  ADAPTIVE  METHODS 


We  present  here  both  the  continuous  and  discrete  forms  of  the  data 

adaptive  methods.  For  either  method,  the  continuous  form  consists  of  a mapping 

from  the  measured  covariance,  K ( t)  |t|  £ T , to  the  spectral  estimate, 

m 

P (f  : T ) | f I <°°.  To  guarantee  the  existence  of  the  map  in  each  case,  we  shall 

m 

assume  that  the  measured  covariance  is  a fixed  segment  of  the  form 
K(t)  = NQ  6(t)  + P(t)  = K*  (-  t);  | t | * Tm 


where  N- is  positive,  |3(t)  is  complex  - valued  and  continuous  on  [-  T , T ],  and 
U mm 

K ( t)  ig  positive  definite.* 

To  develop  the  discrete  form  of  each  method,  we  suppose  that  the 

measured  covariance  is  available  as  2 N 4- 1 samples  of  a continuous  positive 

definite  covariance  segment  on  j~-  T , T^  Jat  uniform  intervals  of  lag.  We  denote 

the  samples  by  the  vector  Y = col  ( Y (-  N),  . • . , Y (-1),  Y(0),  Y(l),  . . . , Y(N) 

where  Y(i)  is  the  covariance  segment  evaluated  at  iA,  and  NA  = T . In  this 

m 

notation,  the  discrete  form  is  a mapping  from  Y to  the  spectral  estimate, 

P(f:Tm),  |f|  <°>. 


*The  positive  definite  constraint  requires  that 


N \m  |cp(x)  | dx+  cp*  (u)  P(u-v)  cp(v)  du  dv 

J0  J0  J0 

be  positive  for  all  square-integrable  functions  cp(x)  on  0,  T^  such  that 


dx>0.  Note  that  13  (t  ) is  not  necessarily  a covariance  function  since  we  have  not 
required  that  it  be  positive. 


7 


The  M.  L.  M.  is  described  in  terms  of  a classical  optimal  filtering  problem. 


The  two  forms  of  this  method  are  readily  obtained  by  considering  continuous  and 
discrete  versions  of  the  optimal  filter. 

The  M.  E.  M.  provides  a certain  positive  definite  extension  of  the  measured 
covariance.  We  develop  the  extension  via  the  concept  of  an  "innovation  process". 

A.  The  M.  L.  M. 

The  M.  L.  M.  spectral  estimate  at  the  point  f Hz  , P_  (f  :T  ),  is  obtained 

o L o m 

as  the  solution  of  a classical  optimal  filtering  problem.  We  consider  the  causal  con- 
volutional filter  of  duration  T seconds,  h (t),  which  satisfies  the  constraint 


and  minimizes  the  average  output  power  when  the  input  is  a zero-mean  random  pro- 
cess whose  covariance  agrees  with  the  measured  covariance  on  interval 


m 


o 


The  M.  L.  M.  spectral  estimate  is  precisely  the  average  output  power 


of  this  filter.  That  is. 


(2.1) 


Where 


T 


m 


hf  (t)  x (t  - t)  d t 
0 o 


(2.2) 


causal  constraint  is  not  essential. 


* 


Since  the  average  output  power  is  independent  of  the  phase  of  H (fQ),  the 
constraint  is  not  essential.  o 
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If  the  solution  for  the  optimal  filter  is  employed  in  2. 1 and  2.  2 to  determine 
the  spectral  estimate  in  terms  of  the  measured  covariance,  the  result  is 

in  -i2nf  t j2nf  u -1  (2.3) 

wv'LSJo e °Q(t’u)e  °dtdu] 

where  Q (t,  u)  is  the  inverse  kernel  of  K (t-u)  defined  by  the  relation 

T 

m 

\ Q(x,  t)  K(t-u)  dt  = 6(x-u);  0<x,  u<Tm  (2.4) 

By  the  assumed  form  of  the  measured  covariance,  the  inverse  kernel  is  well  defined 
Q(t,  u)  = (1/Nq)  [ 6 (t  - u)  -H(t,  u)  ] ; 0 ^ t,  u ^ Tm  (2.  5) 

where  H (t,  u)  (the  Fredholm  resolvent  of  (3  (t  - u)  on  [ 0,  ] ) is  the  solution  to 

the  Fredholm  integral  equation 

T 

m 

N H (t,  t ) + ^ H(t,  u)P  (u-T)du  = p(t-i);  Ost,  tsT  (2.6) 

° ^0  m 

To  indicate  the  discrete  form  of  the  M.  L.  M. , we  define  the  (N  + l)x(N  + l) 
measured  covariance  matrix 

[ %]  = t(j-i)  Is  i,  js  N + l (2.7) 

ij 

and  the  N + 1 dimensional  sampled  frequency  vector 

a i (a  i2nf  A j2nfNA\  „ 

F^  A col  ^1,  e . . . , e J.  (2.8) 
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A discrete  replica  of  the  preceding  optimal  filtering  argument  yields  the 

(2) 

analogous  result 


(2.9) 


We  note  here  that  the  M.  L.  M.  does  not  provide  an  extension  of  the  mea- 
sured covariance.  That  is,  the  inverse  Fourier  transform  of  the  spectral  estimate 
does  not,  in  general,  agree  with  the  measured  covariance  on  the  interval 

r - t , t i 

L m m J 

In  the  next  part,  we  indicate  that  the  M.L.  M.  does  extend  the  measured 
covariance  and  provide  an  interpretation  of  the  extension  in  terms  of  the  innovation 
filter. 

B.  The  M.  E.  M. 

Consider  a zero-mean  stationary  random  process  x (. ) on  [ 0,  °°]  with 
covariance 


E £ x (t)  x (u)  = K E (t  - u)  = Nq  6 (t  - u)  + p R (t  - u)  ; 0 <;  t,  u < ® 


It  is  desired  to  select  p (t)  , | t I <<»  and  a causal  convolutional  filter  h (t  : T ) 

E rn 

of  duration  T seconds  so  that  the  following  constraints  are  satisfied: 


m 


1)  PE  (t)  = P(t)  | t | £ Tm 


2)  The  process  w(.)  on  [T  , ®)  defined  by 


T 


A 


m 


w (t)  = x (t)  - 


_ h(u:  T ) x (t  - u)  d u ; T £ t < °° 
0 m m 
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is  a white  process  for  which 

* 

E [ w(t)  x (u)  ] = 0 fort>u 

The  second  constraint  implies  that 

T 

pm 

E [ w (t)  x*  (t  - t)  ] = Ke  (t)  - \ h (u:  Tm)  Ke(t  - u)  du  = 0 

J0 

0 < T < oo 

Consequently,  P (t)  and  h (t  : T ) must  satisfy  the  Wiener-Hopf  integral  equation 

hi  m 


N h (t  : T ) 
o m 


Since  pg  (x) , 


T 

m 

+ \ h(u:Tm>  P(T"u>du  = Pe^T)’  0st<0°  (2.10) 

h (t  : T ) may  be  obtained  as  the  solution  to  the  integral  equation 


T 

m 

N h (t  : T ) + [ h (u  : T ) 3(r-u)du  = 3(t)  ; 0 £ t £ T (2.11) 

o m J p m m v ' 

Given  h (t  : T ),  it  can  be  shown  that  2. 10  has  a unique  positive  definite  solution, 

* 

K (t),  and  p (t)  is  square- integrable. 

E E 

Conversely,  it  can  be  shown  that  the  functions  p (t  ) and  h (t  : T ) deter- 

E * m 

mined  by  2. 10  and  2. 11  satisfy  the  above  two  constraints  where  the  spectral  height  of 

w (t)  is  N . The  whitening  property  of  the  filter  6 (t)  - h (t  : T ) is  easily  verified 
o * m 

by  computing  E [ w(t)w  (u)  ] and  then  employing  2.10. 


* We  have  obtained  these  results  in  a paper  which  is  to  be  published  - "An  Inno- 

vations Approach  to  Covariance  Extensions.  " 
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The  covariance  Kg  (t  ) is  precisely  the  M.  E.  M.  extension  of  the  measured 
covariance.  Since  the  filter  6 (t)  - h (t  : T ) whitens  the  process  x (. ) in  the  steady 


state  and  the  spectral  height  of  w (. ) is  N , the  M.  E.M.  spectral  estimate  is  given  by 


The  existence  of  the  positive  definite  extension  Kj?(t)  depends  only  on  the  as- 
sumption that  K(t)  is  positive  definite  and  is  independent  of  whether  P(t)  is  positive 
definite.  When  the  positive  definiteness  of  3 (t)  is  assumed,  it  is  of  interest  to  inquire 
whether  3g  (t)  is  positive  definite.  It  follows  from  the  continuity  of  the  integral  term 
in  2.10  that  the  difference,  Nq  h (t:  Tm)  - 3(t),  is  continuous  for  tS  0.  When 


then  3P(T)  is  discontinuous  at  the  point  t=  T . Note  that  3„  (t)  is  continuous  at  the 
e-  m fc, 

origin  by  assumption;  it  then  follows  that  3g  (t)  is  not  a valid  covariance  function 
(see  e.g.,  Papoulis,  Reference  [12  ] p.  334).  It  is  easy  to  demonstrate  cases  where 
3 (T)  is  positive  definite  and  MT^:  Tm)  / 0 (e.g.,  if  3 (t)  = ct>  0,  \ t \ z Tm, 
then  h(T  : Tm)  = a / [ NQ  + a Tm]  . Consequently,  we  may  conclude  that  this  as- 
sumption does  not  imply  that  3r,  (t)  is  positive  definite.  * 

rL 


If  it  is  known  that  the  observed  process  consists  of  a signal  process  plus  an 
uncorrelated  white  process,  then  we  have  additional  information  beyond  the  measured 
covariance.  Therefore,  the  result  does  not  indicate  that  the  M.  E.  M.  generates  an 
"unreasonable"  spectral  estimate  from  the  measured  covariance. 


00 


(2.12) 
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It  is  instructive  to  interpret  the  M.  E.  M.  extension  in  terms  of  the  "innovation 

process  of  x This  process  is  obtained  by  driving  the  causal  linear  filter 

t t 

n(T:  t)  = 6 (t)  - h (r.  t)  (the  "innovation  filter"),  with  x (.  ) . That  is, 

a f 

v (t)  = x (t)  - V h (u  : t)  x (t  - u)  du  ; 0 st  <®  (2. 13) 

Where  h (t:  t)  is  the  solution  to  the  integral  equation 
t 

N h(-r:  t)  + ( h(u:t)  P (r-u)du  = P (t);  0 £ t £ t < <=°  (2. 14) 

° Jq  E E 

It  can  be  verified  that  the  innovation  process  v (. ) is  white  with  spectral  height  N^. 

To  provide  a physical  interpretation  of  the  innovation  process,  we  employ 
[3] 

a result  due  to  T.  Kailath  L J which  indicates  that  we  may  write  x (. ) as  the  sum  of 
a signal  process  s (. ) and  a correlated  white  noise  process  z (. ) where  the  "future" 
noise  is  uncorrelated  with  the  "past"  signal.  That  is, 

x (t)  = s (t)  + z (t)  ; 0 <:  t < c° 

* * 

wnere  E[z(t)z  (u)  ] = N 6 (t  - u) , E[z(t)s(u)]  = 0 

* * 

for  t > u and  p (t  - u)  = E [s  (t)  s (u)]  + E [s  (t)  w (u)]  for  t s u . 

E 

With  this  representation  of  x(.),  the  least-squares  linear  estimate  of  s(t), 
given  the  observation  x (u)  0 £ u £ t,  is  precisely 

A r 

s (t)  = \ h (u : t)  x (t  - u)  du  ; 0 ^ t < °° 

"'0 

J For  notational  convenience,  we  represent  the  innovation  filter  by  giving  its 

response  at  time  t due  to  an  impulse  applied  at  time  t - t. 

| The  observations  of  this  section  concerning  the  innovation  process  apply  more 

generally  to  a stationary  process  y(. ) on  (0,  <*)  with  covariance,  c(t)  = 

No  6 ( t)  + d (t)  , where  c (t  )is  positive  definite. 
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This  indicates  that  h (t  : t)  may  always  be  regarded  as  a causal  least-squares  filter. 
The  innovation  process  then  has  the  interpretation  illustrated  in  Fig.  2. 1. 

Since  the  covariance  of  x (. ) is  the  M.  E.  M.  extension  of  K (t),  the  innova- 
tion filter  is  determined  for  t £ T^  by  the  integral  equation 

t 

N h (t:  t)  + \ h(u:  t)  p (t  - u)  du  = p (t  ) ; 0 £ t £ T (2. 15) 

° J 0 m 

For  t > T , the  innovation  filter  is  given  bv  * 
m 

t:  t)  = 0(t:  T ) = 6 (t)  - h (t:  T ) ; T < t < °°  (2.10) 

m mm 

That  is,  the  innovation  filter  determined  by  the  M.  E.  M.  covariance  K (t  ) is  time 

E 

invariant  for  t > T . Consequently,  the  M.  E.  M.  extension  of  the  measured  covari- 
ance has  a particularly  simple  interpretation  in  terms  of  the  innovation  filter.  For 

t £ T , the  innovation  filter  is  determined  by  the  measured  covariance.  For 
m 

t > T * the  extension  is  chosen  so  that  the  innovation  filter  is  given  by  2. 16. 
m 

This  interpretation  is  readily  applied  to  the  discrete  form  of  the  M.  E.  M.  . 

To  this  end  we  let  x (i)  = 0,  1,  2,  ...  be  a zero-mean  stationary  random  process 

with  covariance 

* 

E [ x (i)  x (j)  ] = Ye  (i  - j)  0 £ i,  j < oo 

where  Y (i)  = 'Ki)  -N  £ i £ N 

E 


* From  2. 10  we  observe  that  h (t:  T ) is  a solution  to  2.  14  for  t > T . Since 

the  solution  is  unique,  the  result  follows. 
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TN-75-7  (2.1) 


l __J 

E [x(t)  x*(u)  ] E [v(t)v*(u)  ] 

= N 6(t-u)+ BE(t-u)  =NQfi(t-u) 


Fig.  2. 1 - The  Innovation  Process, 
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Let  x (n)  denote  the  least-squares  linear  estimate  of  x (n)  given  the  observations, 
x (i),  i = 0,  1,  . . . , n-1,  where  x (0)  = 0.  If  we  interpret  x (n)  as  the  out  put  of 

a causal  linear  filter  - (i),  | the  innovation  process  of  x (. ) is  given  by 

£(n)  = x (n)  - x (n)  = 

11  ni 

x (n)  +^>  Tn(i)  x(n-i)  = ^ ^n(i)x(n-i)  (2.17) 

i = 1 i = 0 

n = 0,  1,  2,  . . . 

where  J 1 i = 0 

n„u)  £ 

/ r (i)  1 ^ i £ n 

1 n 

is  the  innovation  filter. 

We  observe  that  the  innovation  process  is  just  the  estimate  error  as  it  evolves 

in  time.  The  filter  weights  at  time  n are  then  chosen  so  as  to  minimize  the  quantity, 

A * 

= E [ £ (n)  £ (n)  ] (the  average  output  power  of  the  innovation  filter  at  time  n) 

Recall  that  this  is  equivalent  to  requiring  that  the  error  be  orthogonal  to  the  observa- 
tions. That  is,  we  require 

E [ £ (n)  x (n-j)]  = 0 for  j = 1,  2,  ....  n (2.18) 

It  follows  readily  from  2. 17  and  2. 18  that 

Efewe*  <">]  = pn  «tn 

so  that  the  innovation  process  is  white  as  in  the  continuous  case.  The  variance  is, 
however,  time-varying. 

t The  negative  sign  is  chosen  by  convention. 
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Applying  2. 18  to  2. 17,  we  obtain  n + 1 linear  equations  for 
weights  at  time  n: 


p and  the  filter 
n 


tE  <J)  + po  rn  <j)  + 2 yE*  (j 
i = 1 


*>  rn(1> 


pn  = Po  + I Fn  (i)  TE*  (i) 
i = 1 


0 

= 1,  2,  . . . , n 


(2.19) 


In  vector  notation  these  equations  become 

* 


<J> 

— n 


ft 


n 


= P 
— n 


Where  <f>  is  the  (n  + l)x(n  + l)  covariance  matrix,  * . . = 

— n L— n Ji  j 


(2.20) 


v (i  - j),  .£1  is  the  innovation  filter  at  time  n, 
h n 

Uni  - rn(i)' 

1 

and  P = col  (p  , 0,  — , 0) 

— n n 

Analogous  to  the  continuous  case,  we  may  describe  the  M.  E.  M.  extension  of 
the  measured  covariance  in  terms  of  the  innovation  filter.  For  0 ^ n ^ N,  the  inno- 
vation filter  and  its  average  output  power  are  determined  from  the  measured  covariance 
by  2.20.  For  N + 1 £ 00 , the  covariance  extension  is  chosen  so  that 


% “> 


nN(i> 


0 <:  i * N 

; N + 1 £ n < oo 

N + 1 £ i £ n 


17 


/ 


Applying  this  condition  to  2. 19  we  see  that  the  extension  is  determined  by  the  recursive 
relation 


vE  ( j ) = Y g ( - j ) 


N 

‘I  VJ-‘>  V‘> 


j = N + 1,  N + 2,  . . . 


(2.21) 


It  can  be  shown  that  the  sequence  obtained  from  2.  21  is  square-summable  and  positive 

* 

definite  when  is  positive  definite.  It  follows  that  the  discrete  Fourier  transform 
is  well  defined  by  the  limit  in  the  mean 

d ( X ) ^ L.  i.  m. 

E n -» oo  k 

Since  the  convolutional  filter  whitens  the  process  x (. ) in  the  steady  state  and 
its  average  output  power  is  p , we  obtain 


n 

V 

/ 

= - n 


Y ( k ) e 


j 2 rrk  X 


d ( X ) | d Q ( X ) |2 
E N 


N 


(2.22) 


wliere  d^  ( X ) 
N 


rN  (k)e 


- j 2 tt  k \ 


The  M.  E.  M.  spectral  estimate  is  then  given  by 


PE  ■ AV  (At)  - APN 

E 


/ 1 dn 


( Af  ) 


N 


Employing  the  sampled  frequency  vector  defined  in  Part  A,  this  may  be  written  as 

PE  <f:Tm>  = APN/  l^TN  2*n!2  <2'23> 


* The  sequence  is  said  to  be  positive  definite  if  <f>  is  a positive  definite  matrix 

for  n = 1,  2,  . . . n 
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Consequently,  the  M.  E.  M.  spectral  estimate  is  obtained  from  the  discrete 
measured  covariance  by  determining  the  innovation  filter  and  its  average  output  power 
at  time  N from  2.  20.  In  the  next  part  we  indicate  that  the  M.  L.  M.  spectral  estimate 
may  be  obtained  from  the  innovation  filter  and  its  average  output  power  for  times 
1 £ n £ N,  and  demonstrate  a relationship  between  the  M.  E.  M.  and  M.  L.  M. 

C.  A Deterministic  Relationship  between  the  Data  Adaptive  Methods 

In  the  last  part,  we  demonstrated  that  the  innovation  filter  and  its  average 
output  power  at  time  1 £ n £ N is  determined  from  the  measured  covariance  by  the 
vector  equation. 

cf-  ft  = P 1 £ n £ N (2.24) 

— n n — n 

Our  first  step  here  is  to  write  the  M.  L.  M.  spectral  estimate  in  terms  of  the 
innovation  filter. 

Let  x ( i ) i = 0,  1,  2,  . . . , N be  a zero-mean  random  process  with 

covariance 

E[x(i)x(j)]  = Y(i-j)  0 £ i,  j £ N 

and  define  the  vector 

xN  = col  ( x ( N ) , x(N-l  ),...,  x ( 0 ) ) 

If  G N denotes  the  ( N + 1 ) x ( N + 1 ) matrix, 


A 


0 

2Tn-i 

0 

0 

-TN-2 

• 1 
1 1 1 

1 1 1 

1 I 1 

! ' ! 

0 

0 

01 

(2.25) 
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then  the  innovation  process  of  x (. ) may  be  written  in  the  vector  form 


— N 


(2.  26) 


where 
Noting  that 


— N 


= col  ( (n),  G(n  - 1), G(0)  ) 


= E 


[xNXN'T]-d  Dn  - e[£n£ 


*T  1 

N J 


where  is  the  diagonal  matrix 


N 


we  have  from  2.  26 

-N  = — N — N °*N  T 

It  follows  that  the  inverse  measured  covariance  matrix  is  given  by 


(2.27) 


, - 1 * T - 1 

= gki  dk1  g 

-N  -N  -N 


-N 


From  2.  9 we  obtain  the  M.  L.  M.  spectral  estimate 


(2. 28) 


-1  T - 1 * T * T - 1 * 

Pr  (f:  NA)  = F & F = F G D GF 
L K ' -N  -N  N £N  -N  -N  -N  N 


- 1/P0+I«[  E,*|2/P1  +-"+  !nTN  En  |i/pN  <229> 
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By  equation  2.  23  we  may  write 


PE(f:iA)  = AP J IftjV 

where  P„  (f:  0)  = A P 

E 0 . 


2 


l 


0,  1,  2,  ....  N 


It  then  follows  from  2.  29  that 


P ' 1 (f:  NA) 

Li 


A 


l 


PE_1  (f:iA) 


(2.30) 


i = 0 


That  is,  the  reciprocal  of  the  M.  L.  M.  spectrum  is  obtained  by  averaging  the  recipro- 
cals of  the  M.  E.  M.  spectra  obtained  as  the  maximum  lag  of  the  measured  covariance 


varies  from  zero  to  N A . This  result  is  still  subject  to  interpretation.  However, 
it  does  suggest  that  the  M.  L.  M.  spectral  estimate  has  a lower  resolution  capability 
and  is  more  stable  then  the  M.  E.  M.  estimate. 

Equation  2.30  suggests  that  a differential  relationship  exists  between  the 
continuous  forms  of  the  data  adaptive  estimates.  We  now  show  that 


d P ’ 1 (f:  T) 


P'1  (f:  T) 


(2.31) 


dT 


We  employ  2.  3 and  2.  5 to  write  the  M.  L.  M.  spectral  estimate  as 

T T 


This  result  recently  appeared  in  a paper  by  J.  Burg,  Reference  4. 
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Where  H (t,  u:  T)  is  the  Fredholm  resolvent  of  (3  (t  - u)  on  [ 0,  T ],  as  determined 
by  the  integral  equation 


N H(t,  t:T)  + \ H(t,  u:  T)  p(u-i)du  = 0 (t  - t)  ; 0 £ t,  t £ T (2.33) 
U J0 


Differentiating  2.  32,  we  obtain 
N d_  P ~ 1 (f:  T)  = 1 - ( e'  J 2 11  f T 


H (T,  u:  T) 


j 2 nf  u 


du 


dT 


■V  e'j2TTft  H (t,  T:  T)  ej2TTfTdt 
J0 


T T 


-u 


- j 2 TT  f t 


e " 5 H (t,  u:  T)  e' 

0^0  b T 


j 2 nf  u 


d t d u 


(2.34) 


To  evaluate  the  derivative  indicated  in  the  last  term  of  2.  34,  we  use  the  resolvent 
identity  (see  e.  g.  R.  Bellman,  Reference  5) 

* 

H (t,  u:  T)  = -H  (T,  t:  T)  H (T,  u:  T) 


t T 


It  then  follows  easily  from  the  conjugate  symmetry  of  the  Freedholm  resolve  and  a 
change  of  variable  that  2.  34  may  be  written  as 


d P. 


d T 


T 

(f:  T)  = |l  - [ H (T, 
J0 


7,  T - t : T)  e 


~ j 2 n 


f d 1 1 2 /No 


Since  H (T,  T-t:  T)  = h (t:  T)  (Compare  2.11  and  2.33),  2.31  follows  from  2.12. 
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D.  Summary 

In  this  chapter  we  indicated  the  continuous  and  discrete  forms  of  the  data 
adaptive  methods  and  provided  an  interpretation  of  the  M.  E.  M.  extension  in  terms 
of  the  innovation  filter.  We  have  also  shown  that  the  discrete  time  M.  L.  M.  spectrum 
may  be  obtained  from  the  innovation  filter  and  its  average  output  power.  This  result 
is  gainfully  employed  in  Chapter  IV  where  we  consider  a scheme  for  implementing 
the  two  methods  in  which  the  innovation  filter  is  estimated  directly  from  the  observed 
data. 

The  continuous  forms  are  employed  in  the  next  chapter  to  obtain  an  analytical 
comparison  of  the  resolution  capabilities  of  the  two  data  adaptive  methods  and  die 
conventional  (Blackman  - Tukey)  method  when  the  measured  covariance  consists  of  a 
pair  of  tones  in  white  noise. 

III.  A MEASURE  OF  RESOLUTION  FOR  THE  DATA  ADAPTIVE  METHODS 

As  indicated  in  the  introduction,  some  measure  of  estimate  performance  may 
be  obtained  by  applying  the  estimation  methods  to  known  covariance  functions.  In 
this  chapter  we  consider  the  case  where  the  given  covariance  function  is  from  a ran- 
dom process  which  consists  of  two  pure  tones  in  additive  white  noise.  Our  objective 
in  applying  the  estimation  methods  to  this  simple  model  is  to  provide  a quantitative 
measure  of  resolution  which  may  serve  as  a basis  for  comparison  of  the  two  data 
adaptive  methods  and  the  conventional  method.  For  simplicity,  we  consider  the  situ- 
ation where  the  tones  have  equal  power  so  that  the  measured  covariance  is  of  the  form. 

T/-/V  vt  c/  \ i ^ j 2 nf . t 2 j 2 rrf « t i i rp 

K(t)  = N 6 (t  ) + a e 1 +ct  e 2,  t ^T 
0 m 
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Pc(f:Tm)  = D(f)  * [ NQ  + a 6 (f-f^  -f  a2  6(f-y  "] 
= NQ  d(0)  + a2  D(f-f1)  + a2  D(f-y 

= N0  [4  + 6/Tm  (D(f-V  + D<f-f2>)  ] 

From  3.1,  we  obtain 


Rc<9’ 


W)  = [l  + (0/Tm)(D(O)  +D(W/Tm) 


1 + 


(2  e/T  ) 

m 


D (W/2  T ) ] 
m J 


The  locus  of  points  0,  W for  which  the  tones  are  just  separated  is  obtained  by  setting 
R ( 0,  W)  equal  to  one.  This  yields  the  equation 

D(0)  + D(W/T  ) = 2 D ( W/2T  ) (3.2) 

m m 

Equation  3.  2 determines  a "cutoff"  time- bandwidth  product  W which  depends  on  the 
lag  window  used.  For  W > the  tones  are  separated  and  for  W ^ W the  tones 
are  not  separated.  We  have  illustrated  this  in  Figure  3. 1 by  plotting  0' (W)  as  a ver- 
tical line  centered  at  W . The  values  of  for  two  typical  lag  windows  are  shown  in 
the  same  figure. 

Note  that  this  result  is  apparent  from  the  second  term  in  3. 1 since  the  white 
noise  level  appears  simply  as  an  additive  constant  in  the  spectral  estimate.  We  shall 
obtain  a quite  differenct  result  for  the  data  adaptive  methods. 
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TN-75-7  (3.1) 


lag  window 

wc 

rectangular: 

d(T)  = 1 | T |£  Tm 

. 681 

triangular 

d<T>=(-  ^)lTl£Tm 

\ m 7 

.874 

Fig.  3. 1 - 0'  (W)  for  the  Conventional  Method. 
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B.  The  M.  E.  M. 

We  consider  first  the  more  general  case  where  the  measured  covariance  is 


T,/  * k _ . . . \ 2 ] 2 TT  f , T 

K ( T ) = No  6 ( T ) + ) eJ  l 

i = 1 


<:  T 

m 


The  M.  E.  M.  spectral  estimate  is  obtained  from  2.11  and  2. 12.  For  the  above 
measured  covariance,  the  former  equation  becomes 


m 


N h (t  T ) + > ct.  2e  j2rfiT  ^ h (u:  T ) e " j 2 1 u 

0 m / i Jq  m du 

i = 1 


2 j 2nf l t 

o.  e 

l 


0 <;  t <;  T 


m 


(3.3) 


i = 1 


We  let  H (f:  T ) denote  the  Fourier  transform  of  h (t  : T ) . Multiplying  both  side 


• j^TT  f 


m 


of  3.3  by  e and  integrating,  we  obtain  the  equation 


H 


n 

(f : T ) = Y 0 . I"  1 - H (f . : T ) 1 e 

m L i L 1 m J 


j tt (f  . ” f ) T 


i = 1 


m 

sine  („<f . - f)  Tj 


(3.4) 


Where  0.  = c ^ T /1NL  is  the  signal-to-noise  ratio  corresponding  to  the  tone 

i i m 0 r 

and  the  function  sine  ( X ) is  defined  by 

, v v A sin  ( X) 
sine  ( X ) = £ 


We  note  that  3.4  is  a representation  of  H (f:  T ) in  terms  of  its  samples  at  the  n 
tone  frequencies.  To  determine  the  samples,  we  evaluate  3. 4 at  f = f.  for 
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j = 1,  2,  . . . , n.  This  results  in  n linear  equations  for 

H (f . : T ) j = 1,  2,  . . . , n: 

J m 


H (f . 


T ) 

m 


•Ui 


1 - H (f . : T ) 1 
l m J 


sine  [n(f  . - f .)  T ] (3.  5) 

i j m 


j = 1,  2,  . . . , n 

Therefore,  the  M.  E.  M.  spectral  estimate  for  the  general  n-tone  case  may  be  obtained 
from  3.  5,  3.4,  and  2. 12. 

When  n = 2 and  9^  = 9 ^ = 9 , 3.  5 has  the  solution 

1-H(f  :T  ) = 1 ■ H*  (f  : T ) = 
l m 2 m 


(i  + e-  ee 
Equation  3.  4 becomes 


•jnW 

sine 


( TT  W ) )/((*  + 9)2  - e2  sine  2 ( tt  W ) ) (3.6) 


H(f:  T ) 
m 


9^  [l -H(f.:  Tm)  ] e'J  2 n(fi'f)  Tm‘  [sine  Mf.  - f)  TJ  ] (3.7) 
i = 1 


It  follows  from  2. 12  that 

Re(9.  W)  = 1 l-n(fi+f2  ,TJ  |2y/|l-H(f1  :TJ  [2 


(3.8) 


With  some  algebraic  manipulation  3.  8 reduces  to 
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R (9,  W ) = [1  + 0 - 0 sine  (nW)]4  / B 

h 


(3.9) 


where  B = (1  + 0 ) 2 - 2(1  + 0)  Gsinc  (nW)  cos  (nW)  + 0^  sine  ^ (nW). 


Setting  this  term  equal  to  one,  we  obtain  a cubic  equation  for  0'  (W)  : 


(3.10) 


where  a = 1 - sine  (ttW),  * = 1 - cos  (nW), 

T his  equation  may  be  solved  numerically  by  using,  e.g.,  Newton’ s method.  In  Figure 


values  of  the  time-bandwidth  product.  For  larger  values  of  W the  M.  E.  M.  has  the 
same  resolution  as  the  coventional  method  when  the  rectangular  window  is  used  in 


The  plot  indicates  an  important  distinction  between  the  conventional  method 
and  the  M.  E.  M.  . For  the  former  method  the  tones  cannot  be  separated  when 


resolved  for  any  non-zero  time-bandwidth  product  if  the  signal-to-noise  ration  is 


sufficiently  large. 

We  now  show  that  similar  results  hold  for  the  M.  L.  M. 

C*.  ’ The  M.  L.  M. 

To  obtain  the  M.  L.  M.  spectral  estimate  for  the  general  n-tone  case,  we 
employ  2.  4.  This  yields  the  equation 


Bear  in  mind  that  this  result  is  obtained  by  using  a known  covariance  function. 
When  real  data  is  used  to  generate  the  estimate,  the  variance  and  bias  of  the  estimate 
may  preclude  effective  resolution  of  the  tones  when  the  time-bandwidth  product  is  near 
zero. 


3.  2 we  have  plotted  the  solution  on  a dB  scale  (20  log^  (. ) ) for  relatively  small 


tiiat  it  separates  the  tones  for  any  positive  signal-to-noise  ration  if  W >W  — . 681. 


W ^ W . When  the  M.  E.  M.  is  employed,  however,  the  tones  may,  in  principle  be 
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TN-75-7  (3.2) 


Fig.  3.  2 - 0 ’ (W)  for  The  M.  E.  M. 
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m 

n 

,0  X , t)  [n  0 6 (t  - u)  + y a. 


2 j 2 tt  f . (t  - u) 


dt 


i = 1 
T 

n 2 m 


-j2rr  f.(t  - u) 


dt 


= Nq  '2(x,  u)  + Y a.  C Q(x,  t) 
i = 1 J0 

= ^(x  - u) 

Define  the  function 
T T 

A m m .0  r .0  rt 

A r r -j2nfx  j2nf  a 

S (f,  f ) = \ \ e Q (x,  u)  e d x du 

"'0  0 

We  multiply  both  sides  of  3. 11  by  e ^ r^X  e ^ U and  integrate: 

S (f,  f * ) + V 0.  e" J~  (fi  “ f } Tm  sine  [ n(f . - f l ) Tm  ] S(f,  f ) 
i = 1 


= (T  /N  )e 
Am  Cr 


■ JTT  <f  - f’)  T 


m 


sine  ! tt  (f  - f*  ) T 
L mj 


This  equation  gives  S(f,  f ' ) in  terms  of  the  samples,  S(f,  f ) i = 1,  2,  . . 

i 

Evaluating  3.  13  at  f = f j = 1,  2,  . . . , n , we  obtain  n linear  equations 
samples: 


S(  f ; 


11 

, f.)  = y e. 

J J \ 


-jn(f5  - f.)T 


i j m sine 


i = 1 


Or? 


T 


m 


S(f,  f.) 

i 


= (T  /N  J e V Tm  sine  ("tt  (f  - f ) 

V ni  0/  L j 


T 


m J 


J = 1,  2, 


We  note  from  2.3  that  the  M.  L.  M.  speetral  estimate  is 
--1 


P.  (t:  T ) - IS  (f,  f) 

L m L 


(3.11) 

(3.12) 


(3.  13) 

. , n. 
for  the 


n (3. 14) 
(3.  15) 
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Consequently,  3. 14  and  3. 13  determine  the  estimate  for  the  n-tone  case. 
When  n = 2 and  0^  = 0^  = 9,  3. 14  has  the  solution  for  S(f,  f^) 


S(f’  V = (Tm/N0)  6"  JTT  ^ " fl>  ^ { <0+  sinc  l><f  ' fi>  Tm^ 

-9  sine  (rr W sinc  £ n(f  - f2)  TmJ  j (0  4- 1)  - 02  sine2  (nW)| 

S(f,  f^)  is  obtained  by  interchanging  f^  and  f^  in  3. 16. 

Equation  3. 13  then  gives  the  reciprocal  of  the  M.  L.  M.  estimate  in 
the  samples: 

2 

S(f,  f)  = (T  /N„)  - 0 \ e Jn^fi  f)  Tm  sinc  [~  TT(f  * f)  t ~|s(f,  f ) 
m U /_  Limji 

i = 1 

We  note  that 


Rl(9,  W)  = s[(f1+f2)/2,  (f1+f2)/2  ]/«v  y 

With  some  algebra  3. 18  reduces  to 

Rl  (9  , W)  = i (0+  l)2  - 02  sinc  2 (tt  W)  - 2 0 sinc2  (n  W/2)  • 
j"l  + 0^1  - sinc  (tt  W)  ) J | /{l  + 0 (1  - sinc2  (t^W)  )} 


to  one,  we  ODtain 


Setting  R ( 0 , W)  equal 
0'  (W)  = 1 2 sinc2  (ttW/2)  - sinc2  (ttW)  - l}/ 


| 1 - sinc2  (nW)  ■ 2 sinc2  (nW/2)  £ 


1 - sinc 


(TT  W)  ] } 


(3. 16) 
terms  of 


(3.17) 


(3.18) 


(3.19) 


(3.  20) 
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e’(W)  D.  B. 
100.1 


TN- 75-7  (3.3) 


Fig.  3.  3 - The  M.L.  M. 
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This  function  is  plotted  in  Figure  3.  3 over  the  same  range  of  time- bandwidth  product 
considered  for  the  M.  E.  M.  For  larger  values  of  W the  resolution  of  the  M.  L.  M. 
is  identical  to  that  of  the  conventional  triangular  window  in  that  the  M.  L.  M.  separates 
the  tones  for  any  positive  signal-to-noise  ratio  if  W > Wc  =2  . 874. 

We  note  that  the  M.  L.  M.  has  the  same  general  behavior  as  the  M.  E.  M.  in 
that  we  can  theoretically  resolve  tones  which  are  arbitrarily  close  together  if  the 
signal  to-noise -ratio  is  sufficiently  large.  Comparing  figures  3.  2 and  3.3,  we  see 
that  the  M.  E.  M.  requires  less  signal-to-noise  ratio  to  separate  the  tones  than  the 
M.  L.  M.  The  difference  is  about  20  dB  over  the  range  of  W considered. 


IV.  IMPLEMENTATION  OF  THE  DATA  ADAPTIVE  METHODS 

In  Chapter  II  we  indicated  the  mapping  from  the  measured  covariance, 
t(i)  i = -N 0 . . . , N,  to  the  innovation  filter  and  its  average  output  power: 


<T*  0 = P 

— n — n — n 


where 


[♦,]  = T(j-i) 

1J 

P = col  (p  , 0 . . . , 0) 


n = 0,  1,  . . . , N 


i.  j = 1 n + 1 


(4.1) 


It  was  shown  that  both  the  M.  E.  M.  and  M.  L.  M.  spectra  could  be  written 


in  terms  of  ft  . p n = 0,  1 N: 

— n n 

pE(f:Tm>  ■ *pn/IETn2nI2 


(4.2) 


pL(t:  V ■ Oo+  I EiT  Si*  l2/Pi  + +lENT2N*  i'/pJ1  <4- 


3) 
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In  this  chapter,  we  consider  schemes  for  obtaining  the  spectra  from  the  ob- 
served data  segment,  r(t)  0 s t s T.  We  assume  that  r(t)  is  sampled  uniformly  at 
the  M points,  i A i = 0,  1 M - 1,  where  (M  - 1)  A = T,  N <M  and  denote  the 


samples  by  the  vector 
r - col  (rQ,  r1#  . . . 


M 


By  4.2  and  4.3,  our  problem  becomes  one  of  estimating  the  innovation  filter 
and  its  average  output  power  from  the  sampled  data.  We  put  forth  two  general  schemes 
lor  this  purpose  (Parts  B and  C).  With  the  first  (indirect  scheme),  we  compute  the 
measured  covariance  from  the  sampled  data  and  solve  4. 1 for 

, p n = 0,  1,  . . . , N.  The  second  (direct  scheme)  constitutes  a procedure  for 
estimating  Q^,  p n = 0,  1,  . . . , N directly  from  the  sampled  data.  This  scheme 
is  based  on  the  fact  that  the  innovation  filter  weights  are  chosen  so  as  to  minimize 
the  average  output  power  (see  Part  B,  Chapter  II). 

Both  schemes  employ  recursive  relations  for  and  p^  which  we  now  develop. 

A.  Recursive  Equations  for  the  Innovation  Filter 

We  assume  throughtout  this  part  that  the  measured  covariance  is  given  and 
is  positive  definite.  As  in  Part  B of  Chapter  II,  we  denote  the  innovation  filter  at 
time  n by 


— n 

Where  i is  the  least-squares  filter, 
-n 


— n 


To  write  4. 1 in  terms  of  r , we  define  the  n-dimensional  vector 

— n 


Y(l)  - 

v(2) 

- 't'(n)  _ 


(4.4) 


We  have  worked  out  the  details  of  a scheme  suggested  by  Burg,  Reference  S. 
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In  this  notation,  4.1  becomes 
* * 

$ , r = - y 

— n - 1 — n -n 


T * 

p = p + Y r 

n ^0  — n — n 


2,  . . . , N 

(4.5) 

2 N 

(4.6) 

where  p^  = y(0) 


ForY  =col(Y1,  Y2,  . . 
sional  vector  Y given  by 

- = Co1  (Yn’  Yn  - 1’  • • - » Yl) 


, Y ),  define  the  inversion  of  Y to  be  the  n-dimen- 
n — 


(4.7) 


Employing  this  definition  and  4.  5,  we  may  write  the  equations  for  £n  + ^ in 


block  form 


1 

A 

<i>  , 

i 

V 

— n - 1 

i 

— n 

A * T 

i 

Y 

— n 

i 

i 

1 

o 

CL 

B 

— n 
* ' 

Y 


n + 1 


Where 


Y 

— n 


Y 


(n  + l) 


(4.8) 


— n = CGl  <Fn  + 1 (1>>  • 


r . (n)  ) 
n + 1 v ’ ’ 


Yn  + 1 = rn  + l(n  + 1) 


Therefore,  we  have 


$ , B + Y Y , . = - Y 

— n - 1 — n — n n + l — n 


Since  ^ is  a Tbeplitz  matrix,  it  follows  readily  from  4.  5 that 


a a 

- y = <£  , r 

— n — n - 1 — n 


(4.9) 


(4.10) 


Employing  4.  5 and  4. 10  in  4.9,  we  obtain 
* * 

$ .B  =<i>  r +4-  ,.rY,, 

— n-l— n — n - 1 — n — n - 1 — n n + l 


(4. 11) 
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Since  $ ^ is  non-singular,  4.11  yields  the  result 


A * 

b - r + r y 

— n — n — n 

or 


r 

--n  + 1 


n + 1 


(4.12) 


A * 

r r + r Y 

— n — n 

Yn  + 1 


n + 1 


(4.13) 


1 his  equation  is  the  desired  recursive  relation  for  the  innovation  filter.  It  gives  the 
least-squares  filter  at  time  n + 1 in  terms  of  ^ and  the  least-squares  filter  at 
time  n. 


We  now  develop  a recursive  relation  for  the  average  output  power  and  indicate 
an  algorithm  for  obtaining  the  innovation  filter  and  its  average  output  power  from  the 
measured  covariance. 

From  4.  8 we  also  have  the  scalar  equation 


A * ^ * * * 

Y B + pn  Y . = - Y (n  + 1) 

— n — n 0 n + 1 

We  employ  4.12  and  4.6  in  4. 14  to  obtain  the  equation 

V , , = -[~Y  Tr  + Y(n  + 1)  1 /p 

n + 1 L—  n — n J / n 

It  follows  from  4.  6 that 

T * 

^n  + 1 ^0  + — n+1  — n+1  = 


p + Y T B*  + y * Y(n  + 1 ) 
K0  — n -n  n + 1 


(4. 14) 


(4.15) 


(4.16) 
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By  employing  4. 12  and  4. 15  in  4. 16,  we  obtain  a recursive  relation  for  the 


average  output  power: 


Pn  + 1 = Pn(1  ' I Yn  + 1 1 > 


(4.17) 


Note  that  p^  >0  n = 0,  1,  . . . , N,  since  is  positive  definite.  Consequently, 
4.17  implies  that  |y  | |r^(n)  I < 1 n = 1,  2,  . . . , N. 

Equations  4. 13,  4. 15  and  4. 17  constitute  an  algorithm  for  mapping  the  mea- 
sured covariance  into  the  innovation  filter  and  its  average  output  power: 


1)  P0  = m n 0 = [i  ] 

2)  Given  n and  p , we  compute  v . from  4.15,  f2  . and  p . are  then  obtain  - 
ed  from  4. 13  and  4. 17,  respectively. 


B.  The  Indirect  Scheme 

To  implement  the  indirect  scheme,  we  obtain  the  measured  covariance  as 


M-l  , 

f (i)  = ^ ^ r r i = 0,  1,  . . . , N;  N <M 

j = i 

The  innovation  filter  and  its  average  output  power  are  then  computed  via  the  above 
algorithm. 

We  note  that  the  measured  covariance  is  a triangular  weighting  of  the  unbiased 
covariance  estimate. 


M-  i 

K(iA)  = J r.r._4  i = 0,  1,  . . . , N;  N < M 

j = i 

Other  weighting  is  possible,  including  the  rectangular  weighting 

(i.  e.  y(i)  = K(i  A)  ).  The  triangular  weighting,  however,  guarantees  that  the  mea- 
sured covariance  is  non-negative  definite  (see,  e.q.  , Jenkins  and  Watts,  Reference  10). 
This  is  obviously  a necessary  requirement  for  the  data  adaptive  methods. 
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C.  The  Direct  Scheme 


A basic  quantity  in  the  recursive  equations  of  Part  A is  y = r (n),  the 

n n 

response  of  the  innovation  filter  at  time  n ^1  due  to  a discrete  impulse  applied  at  time 
zero.  Since  stationary  filters  are  typically  represented  by  this  response,  we  shall 
refer  to  the  sequence,  -j^,  V ^ • • • » V ^ j as  the  stationary  response  of  the  innova  - 
tion  filter. 

In  Part  A we  observed  that  when  is  positive  definite,  then  the  stationary 
response  at  time  n satisfies  the  constraint 


i 


< 1 


n = 1,  2,  . . . , N 


(4.18) 


It  is  of  interest  that  the  converse  of  this  statement  also  holds.  That  is,  when  p^  is  a 

given  positive  number  and  -j  y , y , . . . , y }- . . ..  , . 

112  NJisa  given  collection  of  complex  num- 

bers which  satisfy  4. 18,  then  there  exists  a unique  positive  definite  Toeplitz  matrix, 

, which  satisfies  4.1  where  Q , p n = 0,  1,  . . . , N are  obtained  from 
-N  — n n 

J "1  * 

1 1*  y2*  * yN  J via  the  recursive  equations,  4.13  and  4.17.  We  note  from 

4. 15  that  the  elements  of  this  matrix  f(j  - i)  = ] » are given  by  the  recursive 

equation 


ij 


T a T 

(n  + 1)  = - p v , + Y T , 
Lnn+1  -n -ft J 


n = 0,  1,  . . . , N-l 


where  v(b)  = p^ . 


Consequently,  any  estimate,  p^,  | y^,  . . . , y^  j-  for  which  p^  is  posi- 


tive and  4. 18  holds  corresponds  to  a positive  definite  measured  covariance.  There  is, 
of  course,  no  need  to  acturally  compute  the  measured  covariance  when  the  stationary 
response  is  obtained  from  the  sampled  data  since  we  may  determine  the  innovation 
filter  and  its  average  output  power  directly  from  4. 13  and  4. 17. 


* J.  Burg  suggested  this  result  in  Reference  8.  We  have  resently  demonstrated 

it  in  a paper  which  is  to  be  published  - "An  Innovations  Approach  to  Covariance  Esten- 
sions. " 
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We  now  estimate  the  stationary  response  via  a recursive  scheme  suggested  by 
Burg.  To  demonstrate  the  scheme,  we  let  ^ be  the  innovation  filter  at  time  k + 1 

determined  by  j y1*  ^2’  ’ ’ ' ’ V Yk  + 1 } where  { V Y2*  ’ * * * \ } is  the  esti" 
mated  stationary  response  up  to  time  k and  y,  is  a complex  parameter  (for  k = 0, 
we  take  fi  = col  (1,  y ) ). 

Our  first  step  is  to  estimate  the  average  output  power  of  0 k + We  let 

denote  the  k + 2 data  points 
A 

Rj  = col  (r  r.  _it  , rj_(k  + j));  k + l^j^M-1 

and  consider  the  power  estimate 


A / 


M - 1 

pk  + i ? v,/2<M-k-1))  l {'e 

j = k + 1 


T „ ,2 

R . + 

k + 1 -j 


A *T 

0.  , , R- 

-k+1 -j 


The  first  term  under  the  summation  sign  is  obtained  by  convolving  £k 


(4.  19) 
with  the  for- 


ward data  sequence  |rQ*  r^,  . . . , r^  ^ jand  the  second  term  results  from  the 


covolution  of  Qk+1  with  the  reverse  data  sequence  -|r 
. . . , . . A 


M- 1*  M 


- 2’  ' ' * ’ r0  }' 


It  is  reasonable  to  take  the  estimate,  v.  ,,  to  be  the  value  to  y.  . which 

k+1  k+1 


minimizes  P 


k + 1* 


To  determine  y. 

Tk  + 1 


, we  use  4.  13  to  partition  Q ^ + ^ 


-k  + 1 


Ik  + l’kA+i. 

Yt 


'k  + 1 


R^.  is  partitioned  as 


R.  = 

-J 


where  X . and  Z . are  scalars. 
J J 


X . 
J 


Y 

-J 


Z . 
J 


39 


With  this  notation,  we  may  rewrite  4. 19  in  terms  of  y 


k + l* 


- 1 


2 (M-K-l)  P. 


T 


k + l 


j = k + l 

* A *x  t * 2 1 

+ | y,  , X . + F i y . + r , I f 

k + l j — k ^-j  — k ^-j  k + l j J 


-*Ty  +y  z I2 

k Yk+1  Yk+1  j1 


{ lxj  + Ek*  Zj  + Ii/  Yi. , a + Y.  • - 2 


(4.  20) 


It  is  easily  verified  with  elementary  calculus  that  the  value  of  y ^ which  minimizes 
this  expression  is  given  by 


- 1 

= -2  )”  (X  +r,Ty.)  (z  + r ry. 

k+l  . L . J -k  *-}'  j k J 


j = K + l 
-1 


;■/ 


r * T * 2 T 2 'i 

lZj  + -k  + V^tc  } 


T 2 


J = k + l 


(4.21) 


This  may  be  written  in  the  form 


A 

Yk  + 1 
Where 


- 1 


= -2 


zk(j>Yk(j) 


J = k+l 


M-  1 


{|Zk<J> 
j = k + l 


+ i Yk<J>  i } 


(4.22) 


A T A a *t 

Zk(j)  = xJ+Ik  Zj.  Yk(j)  = Zj+rk  jr. 

Observe  that  Z (j)  is  the  output  at  time  j when  Q is  convolved  with  the  data  sequence 

K K 


zk<J>  - V2Vt)rj-t 


(4.  23) 


1 = 1 A* 

and  Y (j)  is  the  output  at  time  j - 1 when  Q is  convolved  with  the  data  sequence: 
K K 


V * 

Yk*j)  rj-(k  + l)  + ) rk(/)  r j - (k  + l)  + £ 

-1  = 1 


(4.24) 
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From  4.  22  we  obtain  the  bound 


M-71  M-l 

^k+l1^2/  lZk(j>l  I Yk(j)  1 / J { 1 Zk(j)  1 + |Yk(j)  1 } (4'25) 

j = k+1  j=k  + l 

Since 


°-r  (|zk<j>i  - ' Yk<j> 1 )2  - 

j = k + l 

H-71  r 2 21 

2 {|zk(J"  + lYk(J)i  }•  2 2 I zk<J> ' 

j = k + l j =k  + l 


it  follows  from  4.  25  that 


Y,  , £ 1 
Yk  + 1 

A 


Consequently,  y ^ is  a valid  estimate  of  the  stationary  response  at  time  k + l. 

Since  F,  is  obtained  by  applying  4. 13  to  the  estimate  ( y » y0,  • • • » v, ) for 

K A £t  K 

k ^ 1,  equation  4.  22  constitutes  a recursive  scheme  for  estimating  the  stationary  re- 
sponse of  the  innovation  filter.  From  4.  21,  the  starting  value  for  the  scheme  is 


(4.26) 


It  is  apparent  that,  if  we  implement  4.  22  by  actually  computing  the  convolution 
of  Q,  with  the  data  sequence,  then  the  number  of  complex  multiply  and  add  operations 

XV 

is  prohibitive  for  large  k.  Fortunately,  this  procedure  is  not  necessary  as  we  now 
indicate. 
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By  4.  23  and  4.  24,  we  may  write 


k + 1 


Z,  , 0)  = r + ) r Al)  r = n , R Y,  , (j) 

k + 1 VJ/  j L k + 1 ’ )-l  — k + 1 — j k + 1 VJ/ 

1=  1 


Y 


k -f-1 


(j) 


r j -(k  + 2) 


k + 1 

+ I j - (k  + 2)  + l 
1=  1 


1 


Employing  4.  13  and  the  partition  of  R ^ defined  previously,  we  have 

-k  + 1 -j  Xj  + -k  + -k  Yk  + 1 +Zj  Yk+1  = Zk^)  f Yk+1  Yk(j) 

and 


A*T 

‘k 


A *T 


Q,  R . — x.  ,v,  . + r , y . y . . + r.  y . + z 

-j  - 1 j-1  k + 1 — k ^-j  - 1 k + 1 — k - 1 j-1 


- vk+1  ZkO-‘)  + Yk<J-‘) 


Tlierefore,  we  have  the  recursive  equations 

zk  + i(j)  = Zk(j)+  Yk  + 1 Yk(j)  : k + 2SJ^M-l  (4.27a) 

Yk  + 1(j)  = Y*k  + 1 zk(J-l)  +Yk(j-1);  k + 2 s j ^ M - 1 (4.27b) 

It  follows  from  4.  26  that,  if  we  intialize  4.  27  by  defining 


o 

N 

A 

= r ; 

J 

1 ^ j £ M - 1 

Y„<j> 

A 

= r ; 

J 

1 ^ j £ M - 1 

then  equations  4. 13,  4.  22  and  4.  27  constitute  an  algorithm  for  mapping  the  data  se- 
quence into  the  stationary  response  and  the  innovation  filter. 
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To  complete  our  description  of  the  direct  scheme,  we  need  only  indicate  how 
the  average  output  power  of  the  innovation  filter  is  estimated. 

M-1 

The  logical  choice  for  p = 1/M  S |r  | (4.28) 

J=0  J 

The  remainder  of  the  output  power  sequence  is  obtained  via  4.17 

Pk  + 1 M-  l?k+l  '2)  ; k=0-‘ N-k 

D.  Summary 

In  this  chapter  we  developed  two  basic  schemes  for  estimating  the  innovation 
filter  and  its  average  output  power.  With  the  first  (indirect  scheme)  we  solve  the 
system  of  equations,  4.1,  where  the  measured  covariance  is  taken  to  be  a weighted 
average  of  the  sampled  data.  An  algorithm  for  solving  this  system  of  equations  is 
indicated  in  Part  A.  With  the  second  (direct  scheme)  we  estimate  the  stationary 
response  of  the  innovation  filter,  y^  = (n),  n = 1,  2,  . . . , N,  directly  from  the 

sampled  data  and  determine  the  innovation  filter  via  equation  4. 13.  The  average  out- 
put is  then  computed  from  equation  4.  18  where  p^  (the  covariance  for  zero  lag)  is  es- 
timated by  averaging  the  sampled  data. 

Variations  of  these  two  schemes  may  be  obtained  by  simply  weighting  (window- 
ing) the  respective  estimates.  This  corresponds  to  weighting  the  measured  covari- 
ance when  the  indirect  scheme  is  used  and  weighting  the  estimated  stationary  response 
when  the  direct  scheme  is  used.  We  note  that  the  estimated  stationary  response  may 
be  weighted  in  two  different  ways: 

~ A f A A A 1 

1)  Standard  Weighting:  y^  = w (k)  y^  where  |y^,  y ^ . . . , y^  j is  computed  via 

the  direct  scheme. 

~ A A 

2)  Iterative  Weighting:  Yl_  = w (k)  y , where  y,  is  computed  at  each  iteration  by 
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using  the  weighted  stationary  response,  | Y 2> 


* V, 


to  determine 


r . in  equation  4.  21. 
-k  - 1 


Either  way  yields  another  valid  estimate  of  the  stationary  response  when 


w (k)  j £ 1 


k = 1,  2, 


, N. 


In  the  next  chapter  we  consider  six  spectral  estimates  obtained  by  employing 
the  indirect  scheme,  the  direct  scheme  and  a weighted  version  of  the  direct  scheme 
with  each  of  the  data  adaptive  methods.  A qualitative  evaluation  of  the  relative  per- 
formance is  conducted  by  applying  each  of  the  estimates  to  data  acquired  by  simulating 
a pair  of  tones  in  white  noise. 


v\  application  of  the  spectral  estimates  to  a simulated  random 

PROCESS 


Iii  the  previous  chapter  we  developed  the  indirect  and  the  direct  schemes  for 
estimating  the  innovation  filter  and  its  average  output  power.  It  was  indicated  that 
variations  in  the  latter  scheme  could  be  obtained  by  weighting  the  estimated  stationary 
response  of  the  innovation  filter. 

We  consider  here  the  six  spectral  estimates  determined  by  employing  each  o! 
the  following  schemes  with  the  M.  L.  M.  and  the  M.  E.  M. 

1)  Indirect 

2)  Direct 

3)  Direct-Hanning:  This  scheme  uses  an  iterative  weighting  of  the  estimated  station- 
ary response  (see  Part  D,  Chapter  IV) 

The  weighting  is  the  Hanning  window: 
w (k)  = 1/2  [ 1 + cos  (tt  k/M)  ] k = 1,  2,  . . . , N 

where  M is  the  number  of  data  samples  and  N is  the  largest  time  for  which  the  innova - 

* 

tion  filter  is  estimated. 

We  have  chosen  w (k)  somewhat  arbirarily  to  determine  the  estimate  behavior 
vvlien  a typical  iterative  weighting  is  used. 
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The  spectral  estimates  are  applied  to  data  acquired  by  numerically  simulating 
two  unit  power  tones  in  Gaussian  white  noise.  Our  model  for  the  sampled  data  is  the 
following: 

r = ej2"flnA  + ei2,,f2nA  + V 
n n 


V = V R + J V 1 ; 
n n n 


n = 0,  1,  2,  . . . , M-l 
/ R i \ 

The  2 M random  variables,  (V  , V ) n = 0,  1,  . . . , M - 1,  are  inde- 

\ n n / 

pendent  zero-mean  Gaussian  with  variance  N^/  2 A where  is  the  white  noise  level. 

The  parameters,  (M,  A,  f^,  f^),  are  fixed  at  the  values 

M = 500,  A = . 01  sec. , f . = 10  H . , f „ = 10.  5H  . 

1 z 2 z 

In  Figures  5.1  - 5. 48  we  have  plotted  the  six  estimates  on  a dB  scale 

(10  log  iq  (. ) ) for  a fixed  data  sequence  and  for  various  values  of  the  parameters, 

A A 

9 = M A/  Nq  and  p = N / M.  Figure  5.  0 is  the  index  to  these  plots.  The  plots  of 

the  spectral  estimates  obtained  with  the  Direct -Hanning  scheme  are  not  included  for 
p < .4  since  these  are  nearly  identical  with  the  corresponding  Direct  scheme  plots. 

The  spectral  plots  enable  us  to  provide  a qualitative  comparison  of  the  esti- 
mate performances.  The  primary  measures  of  performance  are  the  estimate  stability 
as  evidenced  by  the  relative  "smoothness"  of  the  estimated  noise  spectrum  (ideally 
the  noise  spectrum  is  flat)  and  the  estimate  resolution  or  ability  to  adequately  depict 
the  presence  of  the  two  tones. 

We  observe  that  the  spectral  plots  support  the  results  of  Chapter  III  in  that 
the  estimate  resolution  improves  with  increasing  signal -to- noise  ratio,  0 = p 0’  . 
Note,  however,  that  the  estimate  stability  decreases  with  increasing  p since  the  num- 
ber of  data  samples  is  fixed.  This  is  most  apparent  with  the  M.  E.  M.  - Direct  esti- 
mate which  is  highly  unstable  for  p = . 5 (seeFigs.  5. 22  and  5.  46). 
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We  have  drawn  the  following  preliminary  conclusions  concerning  the  relative 
behavior  of  the  six  estimates: 

1 ) The  Direct  and  Direct- Hanning  Schemes  give  generally  improved  performance 
over  the  Indirect  Scheme  for  both  data  adaptive  methods.  The  plots  indicate  that  these 
schemes  yield  higher  resolution  than  the  Indirect  Scheme  (see  eq.  Figs.  5.  3 - 5.  4 and 
Figs.  5. 15  - 5. 16).  The  stability  of  the  three  schemes  is  roughly  the  same  except  for 
the  case  where  the  M.  E.  M.  is  used  and  p >0.3.  In  this  case  the  Indirect  and  Direct- 
Tanning  schemes  have  comparable  stability  (see  Figs.  5.  21  and  5.  24). 

2)  The  M.  L.  M.  - Direct  and  M.  L.  M.  Direct- Hanning  estimates  do  not  differ  signifi- 
cantly. 

3)  For  p > .3  the  M.  E.  M.  - Direct- Hanning  estimate  performs  better  than  the 

M.  E.  M.  - Direct  estimate.  This  results  from  the  instability  of  the  M.  E.  M.  - Direct 
estimate  for  p > .3  which  is  apparently  reduced  significantly  by  weighting  the  esti- 
mated stationary  response  (compare  Figs.  5.  22  and  5.  24  and  Figs.  5.  46  and  5.  48). 

Note  that  the  M.  E.  M.  - Direct  estimate  is  relatively  stable  for  . 2 ^ p ^ .3  and  the 
resolution  is  not  appreciably  less  than  that  obtained  for  p > .3. 

4)  The  performance  of  the  M.  E.  M.  - Direct  for  .2  £ p ^ .3  is  comparable  to  the  per- 
formance of  the  M.  L.  M.  - Direct  for  . 4 ^ p ^ . 5 (Compare  eq.  Figs.  5. 10  and  5.  16). 

The  above  conclusions  and  the  additional  consideration  of  computational  speed 
suggest  that  a reasonable  spectral  estimation  procedure  is  to  employ  the  M.  E.  M.  - 
Direct  (or  the  M.  E.  M.  - Direct- Hanning)  estimate  for  .2  £ p ^.3.  We  emphasize, 
however,  that  these  are  preliminary  conclusions  and  additional  testing  is  clearly  re- 
quired. In  particular,  other  weightings  of  the  stationary  response  should  be  considered 
and  the  estimates  should  be  applied  to  a wider  range  of  simulated  spectra. 
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p = .1,  .2,  .3,  .4,  .5 

figs.  5. 1 - 5. 10 

M.  L.  M.  - Direct- Hanning 
p — .4,  .5 

figs.  5.11  - 5.12 

M.  E.  M.  - Indirect,  Direct 
p = .1,  .2,  .3,  .4,  .5 

figs.  5. 13  - 5.  22 

M.  E.  M.  - Direct- Hanning 
p = .4,  .5 

figs.  5.  23  - 5.  24 

6'  = 50 

M.  L.  M.  - Indirect,  Direct 
p = .1,  .2,  .3,  .4,  .5 

figs.  5.  25  - 5.  34 

M.  L.  M.  - Direct- Hanning 
p = .4,  .5 

figs.  5.  35  - 5.  36 

M.E.  M.  - Indirect,  Direct 
p = .1,  .2,  .3,  .4,  .5 

figs.  5.  37  - 5.  46 

M.  E.  M.  - Direct- Hanning 
p = .4,  .5 

figs.  5.47-5.48 

Fig.  5.  0 - Index  to  the  Spectral  Plots . 
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Fig.  5.  1 - Indirect,  p=  . 1. 


48 


(f)  (db) 


TN-75-7  (5.2) 


£Hz  -* 


Fig.  5.  2 - Direct,  p = . 1. 
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Fig.  5.3  - Indirect,  p = .2. 


5b 


(f)  (db) 


10. 

t 


TN-75-7  f5.41 


fHz  -*■ 


Fig.  5. 4 - Direct,  p = . 2. 
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Fig.  5.  5 - Indirect,  p = . 3. 
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Fig.  5.  6 - Direct,  p = . 3. 
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Fig.  5.7-  Indirect,  p = .4. 
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Fig.  5.8-  Direct,  p = .4. 
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Fig.  5.  9 - Indirect,  p = . 5. 
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Fig.  5. 10  - Direct,  p = . 5. 
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Fig.  5.11-  Direct- Hanning,  p = .4. 
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Fig.  5.12  - Direct- Hanning,  p = . 5. 
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Fig.  5. 13  - Indirect,  p = . 1. 
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Fig.  5. 14  - Direct,  p = . 1. 
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Fig.  5.15-  Indirect,  p = . 2. 
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Fig.  5.16  - Direct,  p = .2. 
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Fig.  5. 17  - Indirect,  p = . 3. 
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Fig.  5.18  - Direct,  p = .3. 
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Fig.  5.19  - Indirect,  p = .4. 
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Fig.  5.20  - Direct,  p = .4. 
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Fig.  5. 21  - Indirect,  p = . 5. 
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Fig.  5. 22  - Direct,  p = . 5. 
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Fig.  5.23-  Direct- Hanning,  p = .4. 
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Fig.  5.24  - Direct- Hanning,  p = . 5. 
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Fig.  5.25  - Indirect,  p = . 1. 


72 


(qp)  (j) 


TN-75-7  (5.26) 


5.00  7 00  9.00  11.00  13  00  . 15.01 


Fig.  5. 26  - Direct,  p = . 1. 
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Fig.  5.27  - Indirect,  p = .2. 
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Fig.  5.28  - Direct,  p = .2. 
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Fig.  5.29  - Indirect,  p = . 3. 
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Fig.  5.  30  - Direct,  p = . 3. 
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Fig.  5.31  - Indirect,  p = .4. 
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Fig.  5.32  - Direct,  p = .4. 
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Fig.  5.  33  - Indirect,  p = .5. 
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Fig.  5. 34  - Direct,  p = . 5. 
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Fig.  5.35-  Direct- Hanning,  p = .4. 
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Fig.  5.36-  Direct- Hanning,  p = . 5. 
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Fig.  5.  37  - Indirect,  p = . 1. 
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Fig.  5.  38  - Direct,  p = . 1. 
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Fig.  5.  39  - Indirect,  p = . 2. 
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Fig.  5.40  - Direct,  p = .2. 
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Fig.  5.41  - Indirect,  p = . 3. 
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Fig.  5.42  - Direct,  p = .3. 
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Fig.  5.43  - Indirect,  p = .4. 
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Fig.  5.44  - Direct,  p = .4. 
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Fig.  5.45-  Indirect,  p = .5. 
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Fig.  5.46  - Direct,  p = . 5. 
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Fig.  5.47-  Direct- Hanning,  p = .4. 
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Fig.  5.48-  Direct- Hanning,  p = . 5. 
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